An Example of Unsupervised Networks Kohonen’s Self-organizing Feature Map

Kohonen’s se l f -organizing feature map belongs to a class of unsupervised artificial neural network commonly referred to as topographic maps. It serves two purposes , the quantization and dinlensionality reduct ion of data . A short descr ipt ion of i t s h is tory and i ts b io logical context Ls g iven. We show that the inherent c lass i f icat ion propcrtim of the feature map make it a sui table candidate for s o l v i n g t h e classifjcatjon task in p o w e r s y s t e m a r eas Iikc load forecast ing , faul t d iagnos is and security asscssmento 1. B1OLOGICAJ> MOTIVATION Topographically organized neural maps, like those which inspired Kohonen to develop his self-organizing feature map algorithm, have been observed in warimrs parts of the central nervous system, Cells in primary sensory cortex (that p,art of cor~ex which rcceivcs dircc( sensory input) can be characterized by their receprivc jields. A rcccptive field of a CC1l is that p(art of the sensory world within which an adequate slimulus causes an excitatory or inhibitory response of the cell in question. For instance, cells in primary visual cortex are cxciled selectively by input to a small p(arl of the visual field of the animal, and cells in the sornato-sensory cortex respond prefcrentiall y to a sensa(ion fcl( on a particular p,arl of the skin. Cortical neurons arc not arranged randomly but rather in functional areas. At a smaller scale, within the primary sensory and motor arcm, cells are organized in ordered spatitaJ maps which prcscrvc the (opography of the input space to some extent (“lopogrflphic maps”). For ex,ample neurons in a parlicul{ar ,arca of the somam-sensory cortex will correspond 10 the sensory input of neighboring fingers of the stame had. Starling already in the early seventies [von der Malsburg, 1973; Willshaw and von dcr Malsburg, 1976], there have been numcrorrs attcmpls to untlerskln(t the structure and the formation of such cortical maps by formal modeling While their model requires the postulated existence of so-called marker substances which guide the topographically ordered projection, more recent ctevelopmentat models ‘are based on cm-related electrical activity of input to the cortex. Linsker [1988] proposed a Inrrlti-layered network consisting of linear uniLs in order to model orientation seleclive cells. His work was subscqucnlly revised by Miller et al., [1989] for ocukar dominance columns and orientation columns [Miller, 1994] and explained in terms of principal component analysis. Niebur and Wbrgtitter, [1994], however, have shown that such maps can also be described in terms of very simple geometrical constraints. As discussed in [Ritter, 1988] for somatosensory maps and [Obermayer, 1993] for ocular dominance and orientation columns, the self-organizing feature map introduced in [Kohonen, 1982] provides an elegant and comparatively simple qualitative Inodel which explains all these features with one mechanism. 2. A COMP[JTATIONAL MODEL FOR TOPOGRAPHIC MAPS KOhO1lell [1982] proposed a formal model for the formation and function of topographic maps which he called “topology preserving map” and which is now known as Kohonen’s .~elforgurrizing feature nwp. For a set of input signals, the map is tlesigncd to achieve the following tasks [Kohonen, 1989]: 1) Vector quantization of the input se~ 2) Dimeusionalit y reduction of the input space, 3) Preservation of the topological order present in the similarity relalions of the input vectors In the following sections its laterally connected architecture, winner-take-all processing, and unsupervised learning algorilhm and the resulting properties are discussed. 2.1. Architecture lateral fced)wck through neighborhood reldtions The self-organizing feature map is an array of m processing elemenls (neurons) arranged on a lattice of arbitrary dimension. Most applications use a two-dimensional lattice but models where neurons are arranged on a (one-dimensional) line or in higher-dimensional spaces can be defined. For a given network, the input vectors x have a fixed dimension n. The n components of the input vectors are connected to each neuron in the lattice. A synaptic weight w~ is defined for a com]ecIion from tbe jth component of the input vector to the ith neuron. Therefore, an n-dimensional vector wi of synaptic weights is associated with each neuron i. A neighborhood relationship is specified between the neurons of the Kohonen network. in the biological cortex, the connectivity of neurons dceremes with their relative distance